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Abstract 

Background: Individual family planning service delivery organisations currently rely on service provision data and 
couple-years of protection as health impact measures. Due to the substitution effect and the continuation of users 
of long-term methods, these metrics cannot estimate an organisation's contribution to the national modern 
contraceptive prevalence rate (CPR), the standard metric for measuring family planning programme impacts. 
Increasing CPR is essential for addressing the unmet need for family planning, a recognized global health priority. 
Current health impact estimation models cannot isolate the impact of an organisation in these efforts. Marie 
Stopes International designed the Impact 2 model to measure an organisation's contribution to increases in 
national CPR, as well as resulting health and demographic impacts. This paper aims to describe the methodology 
for modelling increasing national-level CPR as well as to discuss its benefits and limitations. 

Methods: Impact 2 converts service provision data into estimates of the number of family planning users, 
accounting for continuation among users of long-term methods and addressing the challenges of converting 
commodity distribution data of short-term methods into user numbers. These estimates, combined with the client 
profile and data on the organisation's previous year's CPR contribution, enable Impact 2 to estimate which clients 
maintain an organisation's baseline contribution, which ones fulfil population growth offsets, and ultimately, which 
ones increase CPR. 

Results: Illustrative results from Marie Stopes Madagascar show how Impact 2 can be used to estimate an 
organisation's contribution to national changes in the CPR. 

Conclusions: Impact 2 is a useful tool for service delivery organisations to move beyond cruder output measures 
to a better understanding of their role in meeting the global unmet need for family planning. By considering 
health impact from the perspective of an individual organisation. Impact 2 addresses gaps not met by other 
models for family planning service outcomes. Further, the model helps organisations improve service delivery by 
demonstrating that increases in the national CPR are not simply about expanding user numbers; rather, the type of 
user (e.g. adopters, provider changers) must be considered. Impact 2 can be downloaded at http://www. 
mariestopes.org/impact-2. 



Background 

An estimated 222 million women in developing coun- 
tries have an unmet need for modern contraceptives [1]. 
This deficit puts women at risk of unintended pregnan- 
cies, which can result in unsafe abortions or even 
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maternal death. Such a high level of unmet need under- 
scores the pressing global health problem of access to 
modern contraceptive services, an issue the wider repro- 
ductive health and donor community has recognised 
and prioritised. Recently, the July 2012 Family Planning 
Summit announced a commitment to reach 120 million 
additional women in the world's poorest countries with 
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family planning services, in order to reduce the number 
of unintended pregnancies and improve maternal and 
child health [2]. Key benchmarks for global health also 
recognise this unmet need. Increasing contraceptive use 
is one of the chief indicators of Millennium Develop- 
ment Goal 5 which aims to improve maternal health [3]. 
Similarly, the Department for International Develop- 
ment's Framework for Results for improving reproduc- 
tive, maternal, and newborn health monitors the 
number of additional modern family planning users 
reached, with the aim of creating at least 10 million 
more users by 2015 [4]. 

The modern contraceptive prevalence rate (CPR), a 
figure that represents the proportion of women of 
reproductive age (WRA) (15-49 years) using modern 
methods of contraception, serves as the key metric for 
measuring contraceptive uptake in the developing world 
[3]. To increase CPR, family planning programmes must 
augment the proportion of WRA using modern contra- 
ception, in addition to maintaining contraceptive use 
among existing family planning users. The modern CPR 
is comprised of women using short-term methods, such 
as oral contraceptive pills, condoms, and injectables, and 
long-acting and permanent methods (LAPMs), such as 
intrauterine devices (lUDs), implants, and sterilisation. 

Ultimately, increases in CPR will be achieved through 
the joint efforts of a wide range of actors across the 
family planning sector and healthcare systems of devel- 
oping countries, including public and private providers, 
product distributors, and family planning advocates. 
Marie Stopes International (MSI), a United Kingdom- 
based, international non-governmental organisation, 
plays a key role in these efforts. Operating in 42 countries 
throughout the world, MSI's extensive network of clinics, 
outreach teams, and social franchises strengthens health 
systems through its delivery of high-quality reproductive 
health services to underserved women, with an emphasis 
on clinic and provider capacity-building, modern contra- 
ceptive choice, and quality assurance. As part of its rigor- 
ous programme monitoring, the organisation places a 
priority on understanding its contribution towards the 
global goal of achieving universal access to contraception 
[5]. However, as discussed below, attributing national 
level increases in CPR to a single organisation - using a 
metric called "increasing CPR" - is inherently difficult 
due to two key challenges. 

Both challenges derive from the metrics that service 
delivery organisations typically use to measure pro- 
gramme achievement. Most of these organisations track 
programme progress through their service delivery out- 
puts, such as the number of commodities and services 
provided. Organisations also use couple-years of protec- 
tion (CYPs), a metric that weights each family planning 
method according to its effectiveness and the duration of 



protection against pregnancy that the method provides to 
the couple. For example, one 10-year lUD equates to 4.6 
CYPs, a figure based on the lUD's average duration of 
use, while one contraceptive pill cycle provides l/15th of 
a CYP, since 15 pill cycles are needed for a full year of 
protection [6]. To derive the total number of CYPs gen- 
erated by a programme in a year, service provision data 
by method distribution are multiplied by each of these 
factors. While service delivery outputs and CYPs are use- 
ful for understanding the scale of family planning pro- 
grammes, they do not measure how these organisations 
contribute towards increasing CPR. As a result, service 
delivery outputs and CYPs cannot demonstrate an orga- 
nisation's national level health impact. 

The main reason these metrics do not translate into 
national-level impact is the substitution of clients. Some 
clients who are new to a service delivery organisation 
are actually not new to family planning; rather, they pre- 
viously received contraceptives from a different provider. 
Therefore, these clients' move to the new provider can 
increase an individual service delivery organisation's ser- 
vice and CYP numbers without having any effect on 
increasing CPR (Figure 1). A study in Honduras demon- 
strates this substitution effect, finding that increased 
sales of a new, socially-marketed oral contraceptive pill 
had no impact on national levels of contraceptive use; 
prior users were simply substituting their former contra- 
ceptives for the new brand [7]. 

Even if substitution could be addressed, using service 
outputs and CYPs to demonstrate an organisation's 
national impact presents another challenge. In order to 
determine any increase in modern CPR at the national 
level, the number of women using family planning in a 
given year needs to be estimated, a figure not indicated 
by annual service provision data and the resulting CYPs. 
This estimated number is more comprehensive than the 
number of family planning clients served in a given year 
because it takes LAPM clients from past years into 
account. Since LAPMs offer multiple years of coverage, 
women will continue to use these methods in years when 
they did not receive services. While formulas for CYPs 
acknowledge that LAPMs are used for multiple years, 
this metric sums together all of the future years of use 
for a method without defining the time period. For exam- 
ple, a programme generating 1,000 CYPs in 2012 could 
realise all of these in 2012 if they provide short-term 
methods alone, or the programme could produce the 
same number of CYPs over the next 10 years through the 
provision of certain LAPMs. Therefore, CYPs cannot be 
'annualised' to produce an estimate of the number of 
women protected from pregnancy in a given year. To 
determine an organisation's impact on national CPR, 
contraceptive use within a year needs to be isolated, pro- 
ducing an estimate that includes the number of users 
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draws clients from women already using 
contraception from other providers. 
While the organisation sees increases in 
their numbers, there is no national-level 
CPR change. 
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Figure 1 Illustration of substitution effect on service provider's contribution to CPR In this example, an organisation draws additional 
users from women already using contraception from other sources (light blue). Therefore, even though the organisation saw an increase in its 
user numbers (growth in dark blue section), there was no net effect on the national CPR (light blue + dark blue), as it remained at 25%. 



served during that year along with the number of users 
continuing to use LAPMs received in previous years. 

Several models already exist that examine the relation- 
ship between family planning services, CPR, and wider 
health outcomes. Most notably, the FamPlan module of 
the Spectrum modelUng system [8] and the Realty V [9] 
model demonstrate how projected changes in CPR result 
in different fertility rates, allowing estimations of changes 
in other health conditions, such as the number of unin- 
tended pregnancies, unsafe abortions, or maternal deaths. 
Unfortunately, while these models can be used to show 
the impact of CPR increases on reductions in national bur- 
dens of these conditions, they do not cater to the unique 
needs of service delivery organisations. These models can- 
not estimate an organisation's individual contribution to 
increasing CPR, which accounts for the substitution effect. 
Moreover, these existing models cannot transform service 
provision data into increasing CPR by modelling estimates 
of the number of family planning users in a given year. 
Therefore, the national health impact of an organisation's 
family planning programme cannot be determined. 

To fill these gaps, MSI has designed the Impact 2 model. 
This model is a tool that programme managers can use to 
easily and robustly estimate the impact of service provi- 
sion, either past or future, on increasing national CPR. 
Impact 2 estimates the total number of family planning 
users in a given year, and what proportion of these users 
counts towards an organisation's contribution to increas- 
ing CPR among all women, or in-union women as speci- 
fied by the user (fuchsia boxes. Figure 2 below). In doing 
so, the model considers the aforementioned challenges of 



how to account for the continuation of LAPM clients and 
the substitution effect. The Impact 2 model has wide uses 
beyond this function, enabling an organisation to estimate 
its impact using higher-level health, demographic, and 
economic metrics such as unintended pregnancies averted, 
maternal deaths averted, and disability-adjusted life years 
(DALYs) averted (blue boxes. Figure 2). 

This paper describes the methodology behind the first 
level of the Impact 2 model, showing how service provi- 
sion data can be translated into modelled user numbers 
and ultimately, the key model output at its top level: an 
organisation's estimated percentage point contribution to 
increasing the modern CPR. Following the model descrip- 
tion, a demonstration of the model's outputs is presented, 
along with a discussion of the model's benefits and limita- 
tions. The methodology for calculating higher-level mater- 
nal and child health metrics offered by Impact 2 is not 
covered in this paper. 

Methods 

Model overview 

Impact 2 is designed to convert an organisation's service 
delivery data, i.e. the number of services provided or 
commodities distributed to clients, into a range of results, 
including increasing CPR. First, these data are used to 
determine estimates of the number of women served by 
an organisation who are using a family planning method 
during a certain year. To derive these estimates, different 
modelling procedures are used for users of long-term 
and short-term methods due to different characteristics 
of the methods. Then, the model divides these estimated 
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b. Miscarriages averted 

(births X miscarriages per 
pregnancy reaching 27 weeks) + 
(abortions x miscarriages per abortion) 



g. Child deaths averted 

(improved birth spacing) 

live births x previous birth interval 
(PBI) coefficient 



a. Unintended 
pregnancies averted 

users X (40% pregnancy rate - 
method-specific failure rate) 



c. Birth averted 

(live and still) 

pregnancies - miscarriages - 
abortions 



d. Live births averted 

live births x (stillbirth rate/(1000 - 
stillbirth rate) 



h. Maternal deaths 
averted 

live births x (M MR/1 00,000) 
MMR change modelled over time 



i. DALYs averted 

maternal mortality, maternal 
morbidity, and child mortality 



e. Abortions averted 

live births x (abortion 
ratio/100) 



f. Unsafe abortions 
averted 

live births x (unsafe abortion 
ratio/ 100) 



j. Direct cost savings to 
families and 
healthcare systems 

pregnancies x cost per 
pregnancy 

cost includes: antenatal care, 
delivery, post-abortion care, 
treatment of pregnancy, and birth - 
related complications 



J 



Figure 2 Flowchart of model outputs estimated by Impact 2. a. The 40% pregnancy rate is a comparison rate reflecting tine cliance of 
pregnancy liad tine women not been using contraception, b. Miscarriage estimates based on life tables of spontaneous abortion probabilities 
created by Hammerslough (1993) [16]. c/d. National stillbirth rates used are from Cousens et al. (2011) [17]. e/f. National abortion ratios used 
when published; otherwise, sub-regional ratios are used based on WHO and Guttmacher studies [12,18]. g. Methodology developed by 
Population Services International to estimate the incremental effect of birth spacing from large sub-regional demographic health survey datasets 
[19]. These estimates may be unreliable because data about the linkages between CPR, birth spacing, and child mortality are currently very 
limited, and will be improved as more research becomes available in 2013. h. Maternal mortality ratio (MMR) is modelled to change overtime 
based on several point estimates of MMR from WHO [13]. i. Disability-adjusted life years are calculated based on years of life lost (YLL) per each 
maternal and child death, and years of life lost to disability (YLD) from maternal conditions, estimated by applying a sub-regional ratio of YLD/ 
YLL for maternal conditions [20]. j. Costs include supplies and direct labour only; default cost savings assume full coverage (i.e. all women who 
need care receive it), and draw on regional cost and incidence data from the Reproductive Health Costing Tool [21]. 



user numbers among those who will maintain the organi- 
sation's previous year's contribution to the CPR and 
those new users who will be counted towards the percen- 
tage point contribution to increasing CPR. 

The Impact 2 model works at a micro level, an appropri- 
ate approach given it is designed for use by service delivery 
organisations. Impact 2 estimates the outcomes (e.g. unin- 
tended pregnancies averted) for an individual who is using 
family planning provided by a specific service delivery 
organisation. It attempts to isolate the impact of individual 



service delivery organisations, although it does not attempt 
to show indirect population-level changes, such as reduc- 
tions in the total fertility rate (TFR) or the maternal mor- 
tality ratio (MMR). 

Operating in Microsoft Excel, Impact 2 uses a user- 
friendly interface that leads the user through each step of 
the modelling process, from initial inputs to final outputs. 
The model is run on one country (or region) at a time and 
allows the user to look at any trend of the impact of family 
planning services from 2001 to 2020. 
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Note that the Impact 2 model produces results that 
show how an organisation was responsible for adding 
women to the national CPR. Because of the model's 
design, it does not consider if these women would have 
otherwise had access to family planning services from 
another provider. Thus, the results should be interpreted 
as what the organisation contributed to increasing CPR, 
but not how much lower the national CPR would have 
been if the programme never existed. 

Experts at Guttmacher Institute, Futures Institute, 
Population Services International (PSI), International 
Planned Parenthood Foundation, and London School of 
Hygiene and Tropical Medicine have reviewed the 
Impact 2 model. It is freely available for non-commer- 
cial use at http://www.mariestopes.org/impact-2. 

Theoretical basis for dividing user estimates between 
baseline CPR levels and the contribution to increasing 
modern CPR 

In order for an organisation to contribute to increasing 
the modern CPR, it cannot simply rely on counting ser- 
vices provided to women who were not already using 
contraception, called adopters. Increases are only realised 
on top of maintaining an organisation's baseline CPR 
contribution from the previous year. Once this baseline 
level is maintained, adopters can be counted towards 
increasing CPR. 

There are two components involved in maintaining a 
baseline CPR contribution: 1) preserve the organisation's 
absolute number of users at baseline; and 2) reach addi- 
tional users to offset population growth in those coun- 
tries where populations are growing, a reality in most of 
the developing world. Note that population growth 
among women of reproductive age increases the abso- 
lute number of users needed each year to maintain an 
organisation's baseline CPR contribution because CPR is 
a proportional measure. 

Women who are already using a contraceptive method 
from an organisation, and continue to use a method 
from the organisation, will count towards maintaining 
the organisation's baseline number of users. This princi- 
ple also applies to women who decide to switch family 
planning methods, provided they continue to obtain ser- 
vices from the same organisation. Therefore, for each 
year of interest, an estimate of the number of women 
continuing to receive family planning services from the 
organisation is needed. For users of long-term methods, 
continuation is built into the model, so these estimates 
already exist (Figure 3, a). However, because the model 
is based on service provision data and not client-based 
data, it cannot account for: 

♦ Short-term users receiving services from the orga- 
nisation who continue to use contraception in future 



years (the model assumes that all discontinue at the 
end of the year); or 

♦ LAPM users obtaining services from the organisa- 
tion who receive another method after discontinua- 
tion (while service data will capture their resupply 
method, the data will not show their previous family 
planning method). 

To address these shortcomings in the available data. 
Impact 2 uses an estimate of the proportion of clients that 
is continuing to receive services from the organisation as a 
proxy for continuation in these two groups (Figure 3, b). 

In some cases, once all continuation has been accounted 
for, there can still be a gap between the number of conti- 
nuing users and the number needed to maintain the pre- 
vious year's number (Figure 3, c). This gap exists because 
some women may stop using family planning as they no 
longer have a need for contraception, or, they might stop 
visiting the service provider for other reasons. When this 
gap occurs, adopters must fill it. 

Once it is ensured that the previous year's number of 
family planning users is maintained, a portion, and some- 
times all, of the adopters needs to be allocated to main- 
taining the baseline CPR contribution in order to offset 
population growth, if necessary (Figure 3, d). Only adop- 
ters can be counted in this manner because the growth in 
users must be among women not already counted in the 
national CPR. 

Finally, once all gaps in the baseline CPR level are filled, 
any remaining adopters can be counted as contributors 
towards increasing CPR (Figure 3, e). The model uses this 
proportion when determining the service provider's per- 
centage point contribution to increasing modern CPR. 

Note that Impact 2 cannot account for the family plan- 
ning methods delivered by other service providers in a 
country because the model is designed for use by a single 
organisation. Therefore, in order for an organisation's esti- 
mated contribution to increasing CPR to translate into a 
national-level increase, an assumption must be made that 
all other providers maintain at least their baseline CPR 
contributions. If not, an organisation's estimated contribu- 
tion to increasing CPR will help offset a decrease by 
another provider instead. In addition, women who change 
from another provider, called provider changers, do not 
contribute towards maintaining an organisation's baseline 
level or towards increasing CPR, because the CPR already 
included these women. This principle also applies to pro- 
vider changers who switch methods, even more effective 
ones such as a switch from a short-term method to an 
LAPM. Impact 2 employs these principles in order to 
address the substitution of clients, ensuring that service 
delivery organisations are contributing to increasing CPR, 
not claiming a CPR contribution created by another 
organisation. 
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Figure 3 Illustration of the allocation of an organisation's clients for determining its contribution towards increasing CPR. 



Model inputs 

Impact 2 determines an organisation's impact on 
increasing CPR and other health outcomes based on 
various inputs. Service delivery data are the primary 
input. Additional inputs include the organisation's client 
profile and a set of background data and assumptions 
regarding national demographics and maternal health 
outcomes, contraceptive methods used, and costs. As 
with any modelling exercise, the quality of the input 
data will dictate the quality of the results. 
Service delivery data 

The minimum data required to operate Impact 2 are an 
organisation's contraceptive service provision data by 
method and year. The user must enter this data for all 
years of interest. In addition, the model allows for historic 
service provision data (dating back to 1982) to be input so 
that an organisation can understand its impact over time, 
an important model feature given that some women who 
received services in the past may still be using the methods 
during the period of interest. These historic data are used 
to estimate the organisation's baseline contribution to 
increasing CPR, as discussed below. If these historic data 
are not entered into the model, the organisation will not 
be able to estimate an accurate baseline level. 

Client profile 

In order to account for substitution (i.e. existing modern 
contraception users who are new to the organisation), 
and effectively distribute users towards maintaining or 
increasing an organisation's CPR contribution, the 
model relies on a client profile that divides an organisa- 
tion's clients from each year into three groups: 

♦ % adopters: the proportion of clients who were not 
using a modern family planning method before 



receiving services (note: this includes women who 
have used modern contraception in the past (ever- 
use) but were not currently using before coming for 
services); 

♦ % continuers: the proportion of clients who were 
already using a modern family planning method 
which they had received from the service delivery 
organisation; and 

♦ % provider changers: the proportion of clients who 
were existing modern family planning users and who 
changed from a different provider to one affiliated 
with the service delivery organisation of interest. 

The organisation inputs this client profile into the 
model, and can be based on client exit interviews or cli- 
ent registration forms, or estimated based on project 
design. More detailed guidance on how an organisation 
can estimate its client profile can be found in the Impact 
2 training materials available on the MSI website. 

Background data and assumptions 

Impact 2 relies on a series of data and assumptions to 
model the organisation's service data into the various 
results. These data and assumptions include: national 
trends (e.g. population projections, fertility rate projec- 
tions), CPR data, mortality data (MMR, age-specific mor- 
tality rates, unsafe abortion mortality), contraceptive 
method-specific assumptions (e.g. typical-use failure 
rates, median age of sterilisation), pregnancy outcomes 
(e.g. miscarriage rates, abortion ratios), and cost savings 
data (e.g. cost and incidence of pregnancy care and deliv- 
ery). While any values can be entered into the model, 
default data and assumptions have been pre-loaded for 
all developing countries in order to improve the ease of 
use. These default values are based on Demographic and 
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Health Surveys (DHS) [10], United Nations (UN) Popula- 
tion Prospects [11], World Health Organisation (WHO) 
studies [12,13], and numerous other validated sources 
(see Table 1 for sources of data referenced in this paper). 
MSI updates these default data and assumptions annually 
to ensure that Impact 2 models its outputs from the 
most current sources available. 

How the Impact 2 model operates 

Determining estimates of users of long-term methods from 
an organisation 

To estimate the number of clients using a long- acting or 
permanent method of family planning each year. Impact 2 
creates virtual cohorts of users by method and year from 
an organisation s service provision data. Long- acting meth- 
ods are implants and lUDs; permanent methods are male 
and female sterilisation. To estimate continued use in sub- 
sequent years, the model applies annual cumulative conti- 
nuation rates (CCRs) (lUDs and implants) and mortality 
rates (sterilisation) to these cohorts. For users of lUDs and 
implants, the entire cohort is removed from the model 
when the maximum duration of use is reached. Among 
sterilisation users, removal occurs when the cohort is 
no longer of reproductive age, which is assumed to be 
49 years. For male sterilisation, note that the female part- 
ner who is protected by the man's sterilisation is counted 
as the user because measures of contraceptive use are 
based on women using contraception. 

For each long-acting method m (10-year lUD, 5-year 
lUD, 5-year implant, 4-year implant, 3-year implant), the 
number of users in the cohort created in year y is calcu- 
lated for each year {n = 0 to n = maximum duration of 
the method): 

Users-,.„ = Services provided",/^^^^^g^^^^j^_^^^^j^_^^^^ 

where CCR^ is the cumulative continuation rate in 
year n for the specified method. Note that CCRq = 1, 
since all women are using the method at the time of 
insertion. 

For male and female sterilisation, the number of users 
in the cohort created in year y is calculated for each 
year (n = 0 to n = cohort reaches age 49), by applying 
the estimated probability of survival from one year into 



the next. Survival probabilities are not cumulative, 
meaning that the survival rate is applied to the previous 
year s user estimate, rather than the original number of 
male and female sterilisation clients in year y: 

1 

Users""y+n = Users""y x (5p^)5 

where sPx is the probability of survival between ages x 
and x-\-5 for the age group that contains the median age 
of the cohort. The median age of the cohort is set to 
begin at the median age of sterilisation, which then 
increases by one year for each subsequent year. The 
probability of survival is raised to the one-fifth to reflect 
the probability of survival over one year rather than 
across the standard five-year interval for survival prob- 
abilities, assuming an even distribution of survival across 
the age group. The value of sPx is determined based on 
a country's assigned model life table family [14] and the 
projected life expectancy at birth {ej in year y-\-n. When 
determining which model life table to use, MSI applies 
results from its own analyses to assign each country 
into one of nine model life table families: Coale-Demeny 
(CD) East, CD North, CD South, CD West, UN Chilean, 
UN Far East Asian, UN General, UN Latin, and UN 
South Asian. 

Once the number of users for each cohort has been 
determined, Impact 2 calculates the estimated number 
of LAPM users in any given year by totalling the figures 
for each cohort (Figure 4). 

Determining estimates of users of siiort-term metliods 
from an organisation 

Short-term family planning methods are those contracep- 
tive methods that require multiple commodities for one 
year of coverage, such as oral contraceptive pills, con- 
doms, and injectables. For these methods, virtual cohorts 
are not needed because services provided in one year do 
not provide protection beyond that year. Therefore, all 
short-term method users are counted in an organisation's 
annual service provision data for the subsequent years of 
use, even if a woman continues to use a short-term 
method from year to year. 



Table 1 Sources of relevant default assumptions in Impact 2 model 



Data 


Source 


Projected women of reproductive age 


UN Population Prospects (2010) revision [11] 


Projected female life expectancy at birth (eo) 


UN Population Prospects (2010) revision [11] 


Female age-specific mortality rates (sPx) 


Model life table families [14] 


Units needed for one year of coverage 


United States Agency for International Development (USAID) 2011 CYP Update [6] 


Median age of sterilisation 


DHS (for samples greater than 30 women) or weighted regional average [10] 


Cumulative continuation rates (lUD, implant) 


USAID 201 1 CYP Update [6] 
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Figure 4 Illustrative example of determining cohorts of LAPM users. Impact 2 creates a matrix such as the one above for each LAPM used 
by a client of the organisation. Each row represents the cohort that corresponds to the year in which the method was delivered. For example, 
'a' denotes the cohort provided contraceptive services in 1999. This cohort 'a' is shown in subsequent years to represent the users' continuation 
with their LAPM in future years. To estimate the total number of LAPM users of a long-acting and permanent method in any given year, the 
numbers associated with the column of the year of interest are totalled, as shown by the pink box surrounding the cohorts listed in 2005. In 
actuality, the model accounts for cohorts starting from services provided in 1982 to capture all continuation of LAPMs from 2001 (the first year 
of analysis included in the model) onwards. 



The type of data collected by most service delivery 
organisations makes the estimation of short-term users 
difficult, however. In general, providers collect data on 
the number of commodities provided (e.g. number of 
contraceptive pill cycles) rather than the number of cli- 
ents served. For instance, while an organisation's data 
might show the distribution of 13 pill cycles (the number 
needed for a full year of coverage), it is unknown if 13 
different women each received one cycle of pills, if one 
woman received a full year's worth of pills, or if a differ- 
ent distribution was provided. Tracking client numbers 
requires a client-based management information system 
that can link multiple visits in a year to a single client, a 
system that is beyond the resources of most family plan- 
ning providers in the developing world. 

Despite these shortcomings with the data, it is possible 
to estimate the number of users of short-term methods. 
The ultimate aim is to derive an estimate of a method's 
users that is comparable to measures of CPR, which are 
determined by data from national population surveys, 
such as DHS. These surveys have their own limitations, 
however. DHS data capture contraceptive use at a single 
point in time over a given year, with current use' defined 
by the respondent [15]. Depending on when the survey is 
undertaken and how the potential pill clients (anywhere 
from 1 to 13 clients) are distributed across the year (i.e. 
consecutively versus concurrently), it is possible for a sur- 
vey to detect all or none of them. 

For this reason, the Impact 2 model takes a conservative 
approach, counting 'users' as having full coverage for the 
entire year. Thus, in the preceding example, the 13 pill 
cycles are attributed to a single contraceptive pill user over 
the course of the year. Assuming even distribution of use 
across the year, the estimated number of pill users 



captured by the organisation's service delivery data should 
accurately reflect the user numbers captured in the CPR. 
If use were actually skewed (e.g. multiple clients who 
simultaneously take one cycle of pills each), the user esti- 
mates may not accurately reflect the same pill use cap- 
tured by the CPR. 

Therefore, Impact 2 estimates the number of short- 
term method users for each method m as follows: 

users^y = Services provided y/^j^-^g needed for 1 year of coverage™ 

The number of units needed for one year of contra- 
ceptive coverage represents the number of commodities 
needed. This number is similar to the one used to calcu- 
late couple-years of protection (CYP) [6], but it is not 
identical because method failure rates are not included. 
Impact 2 accounts for method failure when estimating 
unintended pregnancies to users. It should be noted that 
Impact 2 assumes that short-term user estimates refer to 
clients who use the short-term contraceptive methods 
according to the general usage patterns applied by 
US AID when calculating CYP factors [6]. Inconsistent 
use that may lead to method failure is accounted for 
elsewhere in the model, when method-specific failure 
rates are applied to estimate unintended pregnancies 
averted (not covered in this paper). 

Determining an organisation's contribution to increasing 
modern CPR 

This section describes each of the calculations underta- 
ken by Impact 2 to estimate an organisation's contribu- 
tion towards increasing modern CPR. It is explained in a 
series of steps. 
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1. Establish a baseline CPR contribution from the 
previous year An organisation's baseline contribution to 
CPR is calculated in terms of its absolute contribution 
(i.e. baseline user numbers) and relative contribution 
(i.e. baseline CPR contribution which accounts for popu- 
lation growth offsets). Both are based on its contribu- 
tions in the baseline year {jq), which is the year before 
the selected trend starts. For example, a trend from 
2005 to 2010 is measured against a 2004 baseline. 

The number of baseUne users counts all users in the 
baseline year, using the methodology described above 
for modelling service provision data into estimates of 
LAPM and short-term method users. The baseline esti- 
mate is made from historic service provision data, which 
the service delivery organisation must input into the 
model: 

Baseline users = Total number of usersy^ 

To estimate the baseline CPR contribution, divide the 
total estimated number of users in the baseline year by the 
projected number of women of reproductive age in the 
same year. If the survey-based CPR estimates entered into 
the model are based on all women, then, the full WRA 
number is used; otherwise, if the CPR estimates are for 
married or cohabitating women (hereafter referred to as 
'in-union ), then the WRA is multiplied by the estimated 
proportion of women in union: 



Baseline CPR contribution 



Total number of users 



Yo 



Number of (WRA or in-union WRA)y 



2. Account for pre-existing LAPM users Pre-existing 
users are the estimated number of LAPM users each year 
that had received LAPM services before the year of inter- 
est (i.e. baseline year and before). These numbers are cal- 
culated by method and year from the virtual cohorts of 
LAPM users created in the baseline year as well as in the 
years prior to baseline for which service provision data 
were entered. The total number of women estimated to 
still be using the long-term method during each year of 
the trend (ji to jy) is summed across each cohort (see 
Figure 4 above). 

3. Calculate the number of new users created each year 
from all family planning methods The service provision 
data entered for the years covered by the trend (ji to j^) 
are converted into the estimated number of new users 
created the year the services are provided, for both 
LAPMs and short-term methods. To calculate this num- 
ber. Impact 2 follows a similar methodology to that 
described above for determining estimates of short-term 
and long-term users. However, the model only calculates 
estimates of LAPM users for the first year. Note that the 



number of first-year LAPM users will be slightly fewer 
than the number of clients served in the first year 
because some women may die, and some women may 
discontinue the method before the end of the first year. 
Below are the formulas that Impact 2 uses for each type 
of contraceptive method: 

♦ For each long-acting method m (10-year lUD, 
5-year lUD, 5-year implant, 4-year implant, 3-year 
implant), the number of users created in year y is 
calculated as follows: 

Newusers"y, = Services provided%,/^^^^^g^ ^^^^^^^^^^^ 

where CCRn is the cumulative continuation rate in 
year n for the specified long-term method. Note that 
CCRq = 1, since all women are using the method at the 
time of insertion. 

♦ For male and female sterilisation, the number of 
users created in year y is calculated as follows: 

1 

New users"^y^ = Services provided"^y^ x (sPx) ^ 

where sPx is the probability of survival between ages x 
and x+S for the age group that contains the median age 
of sterilisation. See above for the source for calculating 

sPx- 

♦ For each of the short-term methods, user numbers 
are calculated by dividing the number of services 
provided by the number of units needed for one 
year of contraceptive coverage: 

New users"^y^ = Services provided jij^^i^^^ needed for 1 year of coverage'" 

4. Distribute first-year users between maintaining 
baseline and increasing user numbers First, Impact 2 
makes an estimate of the total number of users needed 
to maintain the baseline number of users: 

Users needed to maintainy^ = Baseline users - Pre-existing usersy^ 

Then, the model checks to see how the new users 
should be distributed, based on the numbers needed and 
the client profile. To do so, the model initially calculates 
the total number of new users who will be apportioned as 
continuers and adopters: 



New continuers. 



Ti 



New usersy^ x % continuersy^ 



New adoptersy^ = New usersy^ x % adoptersy^ 
There are three possible situations in this apportionment: 
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Situation 1: There are enough new continuers to 
maintain the baseline contribution, and therefore, all 
adopters are allocated to increasing user numbers; 

Situation 2: There are not enough new continuers or 
adopters to maintain the baseline contribution, and 
therefore, all continuers and adopters are allocated to 
maintaining baseline levels, with the number of users 
actually decreasing from the baseline contribution; and 

Situation 3: There are not enough new continuers to 
maintain the baseline number, and therefore, some 
adopters are allocated to maintaining the baseline con- 
tribution, with the remaining adopters allotted to 
increasing user numbers. 

The model checks to see which situation is occurring, 
then distributes new users accordingly, into two groups: 

Maintain usersy^ = New users that maintain the basehne user numbery^ 
Increase usersy^ = New users that increase numbers above the basehne levely^ 

For situation 1, all adopters are allotted to increasing 
user numbers, therefore: 

Maintain usersy^ = Baseline users - Pre-existing usersy^ 

Increase usersy^ = New adoptersy^ 

For situation 2, all adopters are allocated to maintain- 
ing the baseline user number, resulting in: 

Maintain usersy^ = New adoptersy^ + New continuersy^ 

Increase usersy^ = 0 

For situation 3, some adopters will be allocated to 
both maintaining the baseline user number and increas- 
ing user numbers above baseline, so: 

Maintain usersy^ = Baseline users - Pre-existing usersy^ 
Increase userSy^ = New adopterSy^ 

- [Baseline users - Pre-existing usersy^ - New continuersy^ ] 

5. Calculate the proportion of new users counting 
towards CPR in the first year Now, Impact 2 esti- 
mates the proportion of all new users counting towards 
CPR in the first year. To do so, the number of users 
maintaining the baseline contribution and the number 
of users above the baseline level are divided by the 
total number of new users created in the first year. 
Therefore: 

Maintain usersy -i- Increase usersy 



This proportion is applied to the number of new users 
of each method m to estimate the number of users of 
each method that contributes to a CPR increase: 

New CPR users"^y^ = % counting in CPRy^ x New users"^y^ 

6. Calculate the number ofLAPM users who will con- 
tinue the method in subsequent years Once the number 
of new users who count towards CPR is determined, an 
additional step must be taken for the new LAPM users 
counting in the CPR. For these users, an estimate is 
needed of how many will still continue to use their 
LAPM in each subsequent year left in the trend. These 
users are referred to as 'existing users', which are differ- 
ent from pre-existing users (who were provided services 
before the first year of the trend). To determine the num- 
ber of these existing users. Impact 2 follows the metho- 
dology described earlier for calculating estimates of long- 
term users. Thus, for each LAPM method m, the virtual 
cohort of users will be followed, applying the correspond- 
ing continuation rates (based on year) and mortality rates 
(based on age of cohort and life expectancy) each year. 

Note that there are not any existing users in the first 
year (yi). All clients who receive services are counted as 
new users, and any past clients who continue to use 
their LAPM are counted as pre-existing users. 

Z Repeat this procedure (steps 1-6) for the remaining 
years in the trend Impact 2 repeats this process for each 
remaining year in the trend. However, rather than mea- 
suring against the baseline year, calculations are based on 
the year immediately prior to the year of interest. For 
example, the second year in the trend (3/2) measures 
increases against the total number of users contributing 
to CPR in the first year (y^). 

The total number of users contributing to a CPR 
increase in a given year (yj is calculated as follows: 

Total users in CPRy^ = Pre-existing userSy^ 
+ Existing userSy^ + Maintain userSy^ + Increase userSy^ 

Once the total number of users in the CPR in year y^ 
is known, the number of users needed to maintain this 
number of total users in the CPR in the following year 
(yz+i) can be calculated using the following: 

Users needed to maintainy = Total users in CPRy 

/ z+ 1 JZ 

- Pre-existing usersy^^ - Existing usersy^^^ 

As described above in step 4, allocation of new users 
is based on the three scenarios for continuers and adop- 
ters. Thus, new users are distributed accordingly into: 
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Increase usersy^^ = New users that increase the previous year's number 

Next, as described above in step 5, the percentage of 
new users, of both long-term and short-term methods, 
that contributes to an organisation's CPR increase is 
estimated. As noted in step 6, the proportion of LAPM 
users must be input into virtual cohorts, which are 
traced out to estimate the number of existing users in 
each of the trend's remaining years. This iterative pro- 
cess is continued for each successive year, until the end 
of the trend is reached. 

8. Produce the model output: the percentage point 
contribution to increasing CPR The final Impact 2 out- 
put described in this paper is an organisation's estimated 
percentage point contribution to increasing modern 
CPR. In other words, this figure is an estimate of how 
much a service delivery organisation has, or will, 
increase the national CPR from its baseline contribution, 
assuming all other providers at least maintain their 
baseline contributions. 

To estimate this figure, an organisation's total percen- 
tage point contribution to increasing CPR is calculated 
for each year first. To do so. Impact 2 divides the total 
number of users counting in CPR each year (total users 
in CPR in year by the number of WRA in each year 
(or in-union WRA if the CPR of only in-union WRA is 
selected): 



Total % point contributioUy 



Total users in CPRy 



Number of (WRA or in-union WRA)y 



This calculation's result includes the organisation's 
baseline percentage point contribution to CPR (since 
those users who maintain the baseline level are included 
in the total number of users counting in CPR). Therefore, 
to isolate the proportion contributing only to increasing 
CPR, the baseline percentage point contribution must be 
subtracted: 

% point contribution to increasing CPRy^ = Total % point contributiony^ 
- Baseline CPR contributiony^ 



Results 

Actual results from Impact 2 are instructive for demon- 
strating how the model operates. Service provision data 
and a client profile from MSI's field office in Madagascar, 
Marie Stopes Madagascar, were fed into Impact 2 to esti- 
mate user numbers and determine the organisation's 
contribution to increasing modern CPR. Marie Stopes 
Madagascar has been providing reproductive health 
services in this sub-Saharan African country since the 
mid-1990s. In 2011, Marie Stopes Madagascar delivered 
services through 15 clinics, 12 outreach teams, and 133 
social franchise outlets. 



Additional file 1 shows Marie Stopes Madagascar's 
service provision data from 1996-2010. In addition, the 
following client profile was used for all years, based on 
unpublished data from client exit interviews that MSI 
conducted with 438 randomly selected clients in 2011: 

♦ % adopters = 22.4% 

♦ % continuers = 49.2% 

♦ % provider changers = 28.4% 

Using the data in Additional file 1 as well as the 
default assumptions preloaded into the model for Mada- 
gascar, Impact 2 calculated the trends of user numbers 
and CPR increases from 2005 to 2011. Condom users 
were excluded from the calculations to avoid the risk of 
overestimating condom use because of user wastage and 
dual protection (using condoms and another contracep- 
tive method). MSI generally recommends that research- 
ers and programmers calculate contributions to 
increasing CPR without condom use included. 

In 2011, an estimated 200,000 women were using a con- 
traceptive method provided by Marie Stopes Madagascar, 
with nearly 90% of these women using a LAPM (Table 2). 
From 2005-2011, Marie Stopes Madagascar steadily 
increased its family planning service provision, with the 
estimated number of all users increasing eightfold. The 
number of new adopters contributing to increasing CPR 
rose accordingly, also increasing eight times within this 
same time period. The number of adopters increasing 
CPR was slightly smaller than the total number of adop- 
ters served during each year of interest because some 
adopters are allocated to maintaining the previous year's 
(or baseline) CPR contribution. Table 2 shows a complete 
breakdown of the model estimates for different categories 
of users by year. 

Following the methodology described above. Impact 2 
calculated the 2005-2011 baseline CPR and increasing 
CPR contributions from these estimated user numbers. 
The baseline CPR contribution was estimated to be 0.5% 
points; in other words, in 2004, Marie Stopes Madagascar 
provided contraceptive methods to 0.5% of women of 
reproductive age. Beyond maintaining this baseline CPR 
contribution, Marie Stopes Madagascar is estimated to 
have contributed 1.2% points towards increasing the 
national modern CPR from 2005-2011. Due to the design 
of the model, this result can only be interpreted as an 
increase if the assumption that all other providers at least 
maintained their baseline (i.e. 2004) CPR contributions is 
true. 

To show the important role of the client profile in esti- 
mating CPR increases, the percentage point increase to 
modern CPR was recalculated based on two different cli- 
ent profiles: one with a lower percentage of adopters and 
one with a higher percentage of adopters than the actual 
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Table 2 Estimated number of women using family planning services from Marie Stopes Madagascar, 2005-2011, by 
type of user (excluding condoms) 





2005 


2006 


2007 


2008 


2009 


2010 


2011 


Total users 


26,244 


33,525 


46,255 


55,550 


83,317 


141,038 


197,860 


LAPM users 


14,105 


21,371 


31,636 


44,109 


73,427 


123,768 


1 75,283 


Short-term method users 


12,139 


12,154 


14,619 


11,442 


9,890 


1 7,270 


22,577 


Users served each year 


17,617 


20,387 


26,273 


25,708 


41,861 


73,315 


85,573 


Adopters served each year 


3,945 


4,566 


5,884 


5,757 


9,375 


16,419 


19,164 


Adopters increasing CPR each year 


2,489 


3,842 


5,005 


4,676 


8,085 


14,793 


1 7,005 



client profile determined from exit interviews conducted 
in 2011. Running these different scenarios in Impact 2 is 
important when data for each year of interest are unavail- 
able. In this case, Marie Stopes Madagascar only had exit 
interview data from 2011, so the client profile from this 
year needed to serve as a proxy for the other years in the 
trend (2005-2010). As it is unlikely that the 2011 client 
profile accurately applied to these years, it is useful to 
input different client profiles to model a range of possible 
results. As shown in Figure 5, the client profile with the 
highest proportion of adopters yielded considerably higher 
estimates of the percentage point contribution to increas- 
ing CPR. 

To illustrate the importance of accounting for conti- 
nuation among LAPM users from year to year and not 
merely relying on service provision data. Figure 6 shows 
the difference between the number of women served 
with long-term methods and the estimated LAPM user 
numbers modelled by Impact 2. The modelled numbers 



are more than double those tracked by Marie Stopes 
Madagascar's service provision data because Impact 2 
accounts for continuers when determining its estimates 
for users of long-term methods. 

Discussion 

The Impact 2 model offers some important advantages 
for modelling a service delivery organisation's health 
impact in a country as the example from Marie Stopes 
Madagascar demonstrates. Namely, these results show 
that Impact 2's outputs can provide a service delivery 
organisation with a better understanding of its indivi- 
dual contribution to national CPR. Instead of relying on 
service provision data or CYPs that can misrepresent 
programme impact, an organisation can use Impact 2 
estimates to gain a more accurate picture of the results 
of its efforts. These estimates also provide a more 
nuanced understanding of how whom an organisation 
reaches can greatly influence outcomes. 
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Figure 5 Range of Marie Stopes Madagascar's estimated contribution to increasing CPR, 2005-2011, based on different client profiles 
(excluding condoms). Marie Stopes Madagascar's estimated percentage point contribution to increasing modern CPR from 2005-201 1 clianges, 
depending upon tine client profile input into the Impact 2 model. The middle line (blue) shows the model outputs based on the actual profile 
determined by 201 1 exit interviews: 22% adopters, 49% continuers, and 28% provider changers. The bottom line (grey) displays results based on 
a lower percentage of adopters in the profile: 1 1% adopters, 61% continuers, and 28% provider changers. The top line (fuschia) shows the 
model estimates based on a higher percentage of adopters in the profile: 44% adopters, 28% continuers, and 28% provider changers. 
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Figure 6 Comparison of modelled LAPM user numbers with LAPM services provided each year, Marie Stopes Madagascar, 2005-201 1 . 

The number of estimated LAPM users served by Marie Stopes Madagascar is considerably larger than the number of LAPM services tracked in 
its service provision data during a given year. The blue bars represent the number of women that Impact 2 modelled as using a LAPM method 
from Marie Stopes Madagascar in each year. Included in these estimates are pre-existing users of LAPMs who are modelled to still be using the 
method in the year of interest. The grey bars show the number of LAPM services provided each year to Marie Stopes Madagascar clients. These 
service provision data do not recognize continuation among users, something Impact 2 takes into account. 



As shown, estimates of increasing CPR generated by 
Impact 2 isolate the contribution of an individual service 
delivery organisation, a process that other family planning 
models do not currently offer. With Impact 2, an organisa- 
tion can better attribute CPR changes to its specific pro- 
gramme efforts. Moreover, because the analyses can look 
retrospectively, any CPR increase can be benchmarked 
against actual DHS data, obviating the need for the 
assumption that all other providers at least maintained 
their baseline CPR contributions. Through these compari- 
sons, an organisation can interpret its programme impact 
on increasing CPR with reasonable certainty, given the 
organisation is confident in its client profile data. For 
example, in the Madagascar case described above, DHS 
data show modern CPR increasing from 14% in 2004 to 
23% in 2009 [10]. Thus, assuming the client profile is 
accurate. Impact 2 results indicate that Marie Stopes 
Madagascar's service provision contributed around 4% of 
the growth in the CPR. 

Besides isolating a specific organisation's impact, mod- 
elling user numbers with Impact 2 results in richer data 
about programme reach and more robust information 
about programme impacts than measures in current 
use, such as standard service provision data and CYPs. 
As shown in Figure 6, the overall number of LAPM 
users increases at a much faster rate than scale up of 
service provision due to the continuation of LAPM 
users from year to year. In 2011, for example, Marie 
Stopes Madagascar recorded the delivery of 67,111 
LAPM services (Additional file 1), a much smaller figure 
than the 175,283 LAPM users that Impact 2 estimated 
for that year (Table 2). Thus, if an organisation relies 
solely on annual service provision totals for understand- 
ing programme impact, it can underestimate the 



number of users it serves, resulting in lower impact esti- 
mates. While CYPs are able to account for differences 
in usage duration of a contraceptive method, the total 
number of CYPs delivered in a year by a service organi- 
sation cannot be compared to annual impact measures 
such as increasing CPR. The reason why this compari- 
son cannot be made is because CYPs are realised over 
an undefined number of future years, and therefore, this 
metric cannot be determined on an annual basis. Impact 
2 has the added advantage of creating annual figures 
that holistically capture the full impact of an organisa- 
tion's service provision (e.g. number of pregnancies 
averted, number of family planning users). 

An organisation can also overstate its contribution to 
national-level changes if it simply looks at its total esti- 
mated user numbers, without accounting for substitu- 
tion, as well as the dynamics of population growth and 
CPR changes. Impact 2 needed to discount a proportion 
of Marie Stopes Madagascar's estimated users from 
2005-2011 to account for substitution, and allocate the 
majority of the remaining users to maintaining the base- 
line CPR contribution. In each of these years, only 
about 10% of the organisation's estimated users were 
considered as contributors to increasing CPR. If the 
total increase in users from 2005 to 2011, nearly 
200,000 women, was attributed to increasing CPR, 
Marie Stopes Madagascar would have grossly overstated 
its program's contribution to increasing CPR. Using 
Impact 2 helps curb any unrealistic claims of impact. 

The results above also underscore the importance of 
reaching adopters if an organisation wishes to have a mea- 
surable impact on increasing CPR. As explained, only 
adopters are able to contribute to increasing CPR. Figure 5 
demonstrates how increases in the proportion of adopters 
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in the client profile can dramatically affect the estimated 
percentage point contribution to increasing modern CPR. 
For example, if Marie Stopes Madagascar reached twice as 
many adopters, contribution to increasing CPR would be 
nearly doubled. However, these results also demonstrate 
that reaching adopters alone will not translate into CPR 
increases. As shown in Table 2, not all adopters served by 
Marie Stopes Madagascar contribute to increasing CPR. 
Impact 2 must account for maintaining a proportional 
contribution (i.e. percentage of WRA using a method 
from the provider), meaning some, if not all, adopters 
must go towards offsetting population growth. Moreover, 
the model recognises that some existing users will discon- 
tinue and need to be replaced by new adopters. Approxi- 
mately 14% of the adopters from 2005-2011 had to be 
allocated towards maintaining a baseline contribution 
from the previous year. Therefore, it is essential for service 
delivery organisations to concentrate programming efforts 
on continuing to provide services to existing users as well 
as reaching adopters, to ensure that the maximum number 
of adopters is counted towards increasing modern CPR. 
The importance of maintaining this existing contribution 
should not be underestimated, especially in rapidly grow- 
ing countries where much scale up is needed just to keep 
up with population growth. 

Limitations 

While Impact 2 has made large improvements in how ser- 
vice delivery organisations are able to consider their health 
impact, they are still only modelled estimates, which are 
inevitably imprecise. Where applicable, the most 



conservative approach available was taken to minimize 
overstating results. A summary of key limitations is pre- 
sented in Table 3, with additional discussion in the subse- 
quent paragraphs. It is useful to separate limitations due 
to the quality of the input data from those inherent to the 
model design. 

As an organisation's service provision data are the pri- 
mary driver of Impact 2 results, inputting accurate data is 
critical for avoiding the inflation of user estimates. There- 
fore, the service data entered into the model must reflect 
only the services and commodities that reach clients. To 
do so, data collection systems need to be built with 
mechanisms that ensure the quality and accuracy of their 
data, an important underlying principle of all service data 
collection, not just for Impact 2 modelling. In cases where 
an organisation only operates in supply chain stages before 
the direct provision to clients (e.g. procuring contracep- 
tives or selling contraceptives to pharmacies), a discount 
factor may need to be added to account for commodity 
wastage. In this situation, attribution must also be consid- 
ered to avoid double counting. 

The client profile presents another key data input chal- 
lenge for the 'increasing CPR' metric as the methodology 
for estimating CPR increases is very dependent on this 
profile. As noted earlier, if data are not available for all 
years of interest, one year s profile is used as a proxy for 
the other years. Doing so is likely to produce inaccurate 
results because client profiles are expected to change from 
year to year. In addition, how client profile data are col- 
lected is likely to influence results (e.g. exit interviews that 
ask about contraceptive use in the past three months. 



Table 3 Summary of key limitations related to data inputs and model methodology 



Area 


Limitation 


Ability to minimize 


Data-related limitations 


Service provision data 


Service data must reflect services provided directly to 
clients; potential challenges include ensuring data 
quality, and inflation when counting commodities 
further back in supply chain. 


Model is best positioned to be used by organisations 
delivering services directly to clients. Organisations 
should ensure their data accurately reflect services 
provided. 


Client profile data 


High potential to over- or underestimate CPR 
contribution depending on accuracy of client profile 
data. 


High and low estimates created based on alternative 
client profiles. 


National assumptions (e.g. 
mortality rates, discontinuation 
rates, population projections) 


Limited potential to over- or underestimate results. 


Best available data have been pre-loaded, with potential 
error minimised as much as possible. 


Model methodology-related limitations 


Converting services to short- 
term users 


Potential to either over- or underestimate number of 
users due to uncertainty around how many of the 
delivered short-term commodities are actually used. 
For coitus-based methods, there are limited data on 
units needed in a year, and the potential that dual 
protection double counts users. 


Remove condom users from calculations to reduce 
greatest uncertainty. For pills and injectables, the 
approach minimises overestimation as much as possible 
given current data on consistency of use. 


Treatment of provider changers 


Potential to underestimate increases in CPR as model 
assumes provider changers are not replaced, and that 
women leaving the organisation (e.g. women who 
stop using contraceptives) are not counted in the CPR. 


High and low estimates created based on alternative 
client profiles. 
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versus a routine data system). Guidance on developing cli- 
ent profiles is available on MSFs website. These caveats 
underscore the importance of re-running the model with 
different client profiles to understand how its outputs 
change, as shown above in Figure 5. 

Another potential shortfall is the model's treatment of 
provider changers, women who had used a family planning 
method previously and changed service providers. When 
estimating CPR increases, these women are excluded 
because the model sees changing providers as taking mar- 
ket share from another provider rather than growing the 
market. In reality however, there may be a complex 
exchange between providers. Because the model only 
accounts for one organisation, it is unable to determine if 
provider changers were replaced by their previous provi- 
der, or if clients who left the organisation went elsewhere 
or simply stopped using contraceptives. The model 
assumes the most conservative scenario in that all provi- 
der changers' would have continued to receive family plan- 
ning services anyway and that all clients who leave the 
organisation have stopped using contraceptives. This 
assumption means that the model is likely to underesti- 
mate an organisation s contribution to increasing CPR. 
There are situations where these assumptions do not hold 
and it may require a revision of the client profile used in 
the model. For example, if the previous provider has shut 
down, these women may have stopped using contracep- 
tives (and therefore, dropped out of the CPR) had they not 
changed providers. In this case, the organisation may wish 
to count these clients towards maintaining a CPR contri- 
bution, rather than excluding them altogether. To do so, 
the organisation can edit the client profile, with a propor- 
tion of the provider changers added to the proportion of 
continuers. 

Impact 2's reliance on demographic projections from the 
United Nations on fertility rates (e.g. Total Fertility Rate) 
and demographics (e.g. WRA) may also be viewed by 
some as a limitation. Due to the use of these data, the 
model is unable to account for a dynamic relationship 
between increased contraceptive use, fertility rates, popu- 
lation growth, and age structure. However, because Impact 
2 works on a relatively short time frame, the projected 
WRA population used to estimate CPR contributions will 
not be affected by short-term changes in the CPR and 
TFR, due to a lag between the emergence of smaller birth 
cohorts and the time when these cohorts reach reproduc- 
tive age. Therefore, the micro level results from this model 
are still useful and relevant. 

One important limitation of the increasing CPR metric 
itself is that because a woman is counted in the modern 
CPR no matter what contraceptive method she uses, the 
metric does not reflect any added benefit of existing 
users switching to more effective methods. Furthermore, 
in some cases, these switchers are fully excluded from 



the increasing CPR calculation, despite making this 
important change in method, because of the substitution 
effect. For example, a method switcher who changes pro- 
viders in order to acquire the new method is not counted 
as part of the organisation's contribution to increasing 
CPR. While not captured within the increasing CPR 
metric. Impact 2 is capable of modelling the differential 
health impacts that result from the use of more effective 
family planning methods. Other Impact 2 model outputs, 
such as unintended pregnancies averted, factor in 
method switchers who opt for more effective methods. 
Future publications on the Impact 2 model will discuss 
these benefits. 

Finally, many other benefits of providing family plan- 
ning services are challenging, if not impossible, for 
Impact 2 to quantify. For example, the model cannot 
easily assess the role that service quality plays in discon- 
tinuation rates and method failure rates. Quantitative 
data also cannot describe how improved birth spacing 
enhances a mother's and a family's well-being, or what 
structural barriers exist in the country to prevent 
women from accessing the contraceptive services they 
need. Therefore, results from Impact 2 should always be 
placed within the national context, and supplemented 
with additional data from qualitative studies and other 
sources. 

Conclusion 

With Marie Stopes International's Impact 2 model, ser- 
vice delivery organisations are better equipped to plan 
for and demonstrate their progress towards key global 
health goals, such as reducing the unmet need for family 
planning. By considering health impact from the per- 
spective of an individual service delivery organisation. 
Impact 2 fills the gap not met by other models for 
family planning service outcomes. With this model, an 
organisation's service provision data can be converted 
into more useful measures of impact than simply count- 
ing the number of services and commodities provided, 
the number of clients served, or the number of couple- 
years of protection afforded. It does so while accounting 
for complex issues germane to family planning service 
delivery and essential to consider when developing effec- 
tive metrics to measure impact: 

♦ First, maintaining the existing CPR cannot be over- 
looked when estimating a service delivery organisa- 
tion's contribution to increasing national CPR. 
Metrics that only look at adopters reached, without 
accounting for the nuances of population growth and 
maintenance of previous usage levels, will not fully 
capture the impact of services on changing CPR. 

♦ Second, the model stresses the importance of 
accounting for substitution, i.e. providing services to 
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women who were already using family planning ser- 
vices from another source. While a scale up in ser- 
vice delivery numbers will translate to a scale up in 
total impacts, this increase may not transfer into lar- 
ger national contributions, however. Metrics that 
only capture scale (such as service provision num- 
bers or CYPs) are unable to address substitution. 
♦ Last, the model incorporates the issue of continua- 
tion among LAPM users into an annual output mea- 
surement. Metrics that only consider clients served 
within the raw data's year of interest do not fully 
capture program impact in that year. 

Thus, MSFs Impact 2 methodology allows organisa- 
tions to move beyond cruder output measures to a bet- 
ter understanding of the role they play in the countries 
where they work. 

Impact 2 also offers users the opportunity to model a 
wide range of results, of which percentage point contribu- 
tion to increasing modern CPR is just one. While not dis- 
cussed in this paper. Impact 2 is capable of producing a 
series of metrics related to the estimated health, demo- 
graphic, and economic impacts of family planning services 
that are recognized as standards by the global health com- 
munity. Such results are useful within many different con- 
texts, including programme and policy advocacy, 
communications with governments and donors, and mon- 
itoring programme progress over time. Moreover, Impact 
2 enables both retrospective and prospective modelling, a 
capability that empowers organisations with informed 
decision making and more realistic programme planning. 

Impact 2 results can also play an important role in 
improving service delivery. The model structure and the 
estimates produced highlight the need for an organisation 
to carefully consider whom it is reaching (e.g. adopters, 
provider changers). Doing so ensures that its programme 
is playing a role in growing the market, not just increasing 
user numbers. As service providers answer the recent call 
to address the large unmet need for family planning 
worldwide, knowledge of how to accurately measure this 
expansion will be indispensable. 

Additional material 



Additional file 1: Service provision data from Marie Stopes Madagascar, 
1996-2011, by numbers of commodities/services provided by method. 
TInese data sinow Inow many family planning commodities and services 
Marie Stopes Madagascar provided to clients from 1996-2011. They are 
organised by method. 
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